Different kinds of interchangeable methods in multitrait-multimethod analysis: a note on the multilevel CFA-MTMM model by Koch et al. (2014)

Koch et al. (2014) extend longitudinal models of confirmatory factor analysis (CFA) for multitrait-multimethod (MTMM) data (e.g., Geiser and Lockhart, 2012) to accommodate different sorts of methods, that is, structurally different and interchangeable methods. While structurally different methods conform to sampling schemes in which each target (e.g., individual employee) is linked to a unique source of information per method (e.g., self, superior, spouse), the sampling schemes of interchangeable methods follow a random selection of sources from a larger set (e.g., three randomly chosen colleagues). Effects of structurally different and interchangeable methods can be jointly analyzed in the framework of multilevel CFA, where targets and structurally different methods are modeled on Level 2 and interchangeable methods are modeled on Level 1 nested under targets (see Eid et al., 2008). In this commentary, we focus on the effects of interchangeable methods in the longitudinal multilevel CFA-MTMM model by Koch et al. (2014) and argue that the model assumptions may be violated in scenarios with more complex hierarchical data structures. In the longitudinal CFA-MTMM model with interchangeable methods, the observed value in indicator i of construct j assessed by method k at occasion l for a given combination of target t on Level 2 with a randomly chosen rater r on Level 1 is specified as